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NESC Position Paper 


Request Number #: 03-004-E 


Requestor Name: Matt Landano (JPL) 

Requestor Contact Info: (818)-354-5624 

Short Title: Mars Exploration Rover (MER) Flight Operations 

Description: NESC Human Factors experts to participate in pre-landing MER Operations 
Readiness Review; NESC Flight Sciences experts to serve on MER Entry, Descent and 

Landing Red Team 


Date Received: 09-23-03 

Date Consultation Initiated: 12-03-03 

Lead Assigned: Dr. Dave Leckrone 

Lead Contact Info: (301)-286-5975 

Date Consultation Concluded: 03-26-04 


Background: 

The Mars Exploration Rover (MER) Project at the Jet Propulsion Laboratory developed 
two golf-cart size robotic vehicles, Spirit and Opportunity, for geological exploration of 
designated target areas on the surface of Mars. The primary scientific objective of these 
missions was the search for evidence of the presence of water on or near the surface of 
the planet during its history. Spirit and Opportunity were launched on June 10 and July 7, 
2003, with their respective landings scheduled for January 4 and January 25, 2004 
(UTC). NASA views the MER missions as particularly critical because of their scientific 
importance in the ongoing search for conditions under which life might have existed 
elsewhere in the solar system, because of their high level of public interest and because 
more than half of all prior missions launched to Mars internationally have failed. 

The development, flight operations and surface operations of the two MER vehicles 
presented many challenges to the JPL Project, two of which motivated requests for NESC 
consultations - the critical entry, descent and landing (EDL) phase of flight operations, 
and Human Factors associated with ground operations of the vehicles after landing. The 
EDL phase consisted of a complex sequence of autonomously commanded events 
intended to decelerate the MER from approximately 12,000 mph to a soft touchdown 
over the course of approximately 6 minutes. The sequence begins with direct entry from 
its interplanetary trajectory, with the aero-shell’s heat shield reaching a maximum 
temperature of 2637 degrees F. This is followed by deployment of a braking parachute 
and heat shield jettison; separation of the Lander vehicle from the back shell, with the 
Lander descending on a tether or bridle; inflation of the airbags needed to provide a soft 
and protected landing of the MER vehicle cocooned within; firing of three solid retro 
rockets bringing the Lander to zero vertical velocity approximately 40 feet off the 
Martian surface; cutting of the bridle, allowing the Lander to free fall to the surface; and 
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multiple bounces of the airbags across the Martian surface with the Lander ultimately 
rolling to a stop. A “Reconstruction” Team within the JPL MER Project was assigned the 
task of piecing together the EDL performance of Spirit with the intent of recommending 
any needed modifications to the sequence for Opportunity, to enhance its chances of a 
successful landing. As an additional risk-reducing measure, an independent EDL “Red 
Team”, comprised of independent outside experts was assigned the task of monitoring 
and assessing the reconstruction activities of the Project team. 

After landing, each MER had a nominal prime mission period of 90 days, with the 
possibility of subsequent mission extension for as long as the vehicles’ batteries and other 
subsystems remained viable. The MER ground control team consisted of engineering 
personnel responsible for the safe and successful operation of the Spirit and Opportunity 
vehicles and instruments on a 24 hour-per-day basis. In parallel, teams of scientific 
investigators developed and maintained the exploration strategy that informed operational 
commanding of the two MER’s. Vehicle operations were closely coupled to the 
availability of sunlight on the Martian surface needed for re-charging batteries and to the 
schedules of communication opportunities. Consequently, ground operations were tightly 
locked to the Martian rotation period or day (called a “Sol”), approximately 40 minutes 
longer than an Earth day in duration. Work shifts of 10-12 hours (or occasionally longer) 
continuously slipped in start and stop time by 40 minutes per Earth day. Thus, the effects 
of long work shifts in a high-pressure environment were exacerbated by the progressive 
loss of synchronism with the patterns of daily life outside the JPL control center. The 
potential for operator error resulting from fatigue or stress under these demanding 
circumstances, as well as concerns about the health of the operations staff, motivated the 
MER Project to seek remedies to minimize the risk and to optimize team performance. 


Request to NESC for Technical Consultations: 

In September 2003 the NESC Chief Engineer at JPL forwarded requests to the NESC 
Board for technical experts in the areas of Flight Sciences (Entry, Descent and Landing) 
and Human Factors to participate in the MER review processes at JPL. Formally, under 
NESC procedures, this was a request for “Technical Consultation.” The NESC Director 
designated the NESC Chief Scientist to take the lead in coordinating this consultation. 

The Chief Scientist requested support from the NASA Discipline Expert (NDE) for Flight 
Sciences and the NDE for Human Factors. In addition the NESC Chief Engineer at ARC 
suggested the possibility of Flight Sciences support from that center. 
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The following experts were identified as leading authorities with the requisite experience 
in these disciplines and they agreed to serve on the MER Technical Consultation Team: 

Human Factors: Dr. Cynthia Null, ARC 

Dr. John Caldwell, IJSAF, Brooks Air Force Base 

Flight Sciences: Mr. Claude Graves, JSC 
Dr. Dean Kontinos, ARC. 

Brief biographies of these individuals are provided in Appendix A. 

Dr. Null and Dr. Caldwell participated in the MER Operations Readiness Review (ORR), 
December 3-4, 2003. Their findings and recommendations were included in the ORR 
Board report to JPL management and are documented in their individual reports provided 
in Appendices B and C, respectively. Dr. Null paid a follow-up visit to JPL on March 3, 
2004 to assess how the ORR recommendations had been implemented. Appendix B 
includes a summary of her observations from that visit. 

Mr. Graves attended the MER Project Operational Readiness Test, December 6-9, 2003, 
to observe firsthand a simulation of the MER landing and the EDL process and 
procedures. The independent EDL Red Team convened at JPL to witness Spirit’s entry, 
descent and landing on January 4, 2004, and met each day thereafter through January 8, 
to assess the EDL reconstruction activities in the MER Project Reconstruction Team’s 
“war room”. The Project Reconstruction Team divided itself into sub-teams focusing on 
specific aspects of EDL. The Red Team assigned one or more of its members to each of 
these sub-teams. Mr. Graves worked with the Entry Dynamics sub-team, while Dr. 
Kontinos was assigned to Descent Dynamics, Deployments and Separations. The full Red 
Team convened daily to observe the full Reconstruction Team tag-up meetings. Mr. 
Graves and Dr. Kontinos fully participated on the Red Team and contributed to its final 
report to JPL management. Appendices D and E, respectively, reproduce their individual 
reports. Appendix F provides the Red Team final report in its entirety. 

The following sections summarize and discuss the findings and recommendations of the 
NESC MER Consultation Team in these two areas. 

Findings and Recommendations: 

1 . Finding : There is inadequate awareness within the Agency of established 
standards regarding acceptable workload and fatigue mitigation. Little leverage 
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currently exists to apply such standards to individuals or groups who are not 
NASA employees or contractors. 

Recommendation : NASA should implement and assure the work time limits 
for critical operations across the Agency as outlined in NASA Procedural 
Requirement 1800.1. Means should be found to apply such standards to all 
parties, whose performance is important to mission success, even those not 
employed by NASA or its contractors. 

2. Finding : Lack of in situ measurements of atmospheric properties actually 
encountered during entry and descent forces the EDL Reconstruction Team to 
rely on assumptions and inferences in diagnosing potentially serious deviations 
from expected vehicle performance. 

Recommendation : We strongly recommend that pressure and temperature 
sensors (or some form of an atmospheric data system) be carried on the 
spacecraft in all future missions that include EDL and that do not have an 
independent method for acquiring critical in situ atmospheric parameters. 

3. Finding : The overall EDL reconstruction process is not well documented, 
making it very difficult for an oversight group such as the Red Team to penetrate 
and assess. 

Recommendation : The EDL technical processes, as well as the processes and 
schedules for required output products and key decisions, should be well 
documented in a form that is usable by independent experts. 

4. Finding : The occurrence of higher than expected angle-of-attack and large- 
amplitude oscillations of the entry vehicle below Mach 6 have not been explained. 

NESC Recommendation : Follow-up investigation to ascertain the causes of 
dynamic instability and high angle-of-attack is important for future 
planetary missions and should be pursued by JPL. The results of this 
investigation should formally be reported in a widely available professional 
publication. The NESC Flight Sciences Super Problem Resolution Team 
should review and assess the results. 

5. Finding : Specific displays and software tools would strengthen the 
reconstruction teams’ ability to properly understand EDL events and to draw valid 
conclusions and make recommendations in a timely manner. 

Recommendation : JPL should provide to the Reconstruction Teams for 
future EDL’s software tools to visually display multiple variables on the EDL 
timeline and to perform automated frequency decomposition of the data. 
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Human Factors Discussion: 


The MER Project clearly recognized the importance of Human Factors in creating the 
environment necessary for mission success. The Project, working with the Fatigue 
Countermeasures Group at Ames Research Center, took a number of steps in the years 
and months leading up to the ORR to address the potential problems. These included: 
incorporating in the operations staffing plan the provision for a 4-days-on/3-days-off 
schedule for staff working 10-12 hour shifts; providing on-site napping facilities for use 
in situations where personnel must remain at JPL for excessively long duty periods; 
providing sleeping quarters at a nearby hotel for individuals who might be too fatigued to 
drive home; providing on-site food, medical and other services at the “odd” times 
dictated by the work schedule linked to Martian time; providing training to managers and 
staff members regarding causes, potential consequences and amelioration of fatigue; 
making special housing arrangements off-site for members of the science teams to 
provide relative isolation from sleep-disrupting factors. And watches that keep Martian 
time were supplied to staff members. 

The assessment by the NESC Human Factors experts noted several areas in which 
mission preparations could be improved. There was a general impression that the MER 
leadership might be underestimating the potential consequences of fatigue to personnel 
safety and effectiveness. In particular there was a concern that the leadership might place 
too much faith in their own ability to monitor the fatigue level of individuals and to gauge 
accurately the extent to which staff fatigue was threatening day-to-day mission execution. 
There appeared to be no “hard-and-fast” rules limiting the number of hours per day or the 
maximum number of days per week or per month beyond which personnel simply would 
not be allowed to work. At the time of the ORR, neither the MER Project, nor the NESC 
Human Factors experts were aware of the relatively new NASA Procedural Requirement 
NPR 1800.1 (October 16, 2002), pages 202-209, that sets objective criteria for maximum 
durations of work before a break is required. Consequently, there were no formalized 
procedures in place for identifying when the workload on individual employees or groups 
had become so excessive as to constitute a threat to mission success, nor plans for safely 
handling unavoidable increases in workload. The fatigue training offered for all MER 
staff members was apparently made optional. Although the impulse against mandatory 
training is understandable, it does disregard the fact that improperly managed fatigue has 
been identified as a causative factor in several high-profile disasters, including the 
Challenger mishap. No provision had been made for frequent, independent assessments 
of fatigue levels in MER personnel beyond simply relying on the subjective judgment of 
the team leaders, who may themselves be overly fatigued or whose judgment may be 
swayed by other considerations (e.g. availability of critical individuals, avoiding 
unanticipated staffing shortfalls, etc.). Stress and fatigue issues were likely to emerge 
even prior to surface operations with last-minute pressures to resolve sequence 
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development and testing problems. It was not clear to the NESC experts that the prior 
experience with the Mars Pathfinder mission, that surface operations would entail a 
decreased workload and would be less stressful than development, was as applicable as 
MER managers believed. Significant development work and system “de-bugging” 
remained to be done in the weeks leading up to EDL and Impact-to-Egress, and the 
associated stresses could readily carry over into the surface operations phase. 

The follow-up visit to JPL by Dr. Null in February produced an important observation 
about stress and fatigue of the members of the science teams. Although MER Project 
management was able to implement work limits (10-12 hour days, 4 days work followed 
by 3 days off) for the civil servant and contractor staff members responsible for the 
operational safety and performance of the MER vehicles on Mars, little leverage existed 
to require the science teams to pay attention to their own fatigue levels. The Opportunity 
Science Team was “worked to the bone” to prepare for a press conference in Washington, 
D.C. on March 2. The scientists who flew to the east coast to appear at the press 
conference apparently resumed work almost immediately upon returning to JPL without 
much of a break. Members of the Spirit Science Team who were interviewed admitted to 
being exhausted, but were unwilling to leave their work. This is cause for concern. 
Although the science teams had no direct operational responsibility for the “health and 
safety” of the rover vehicles, they were responsible for defining on a day-by-day basis the 
scientific strategy that guided mission operations. Such decisions must be made with 
clear heads. Poor judgment and poor decisions arising from fatigue could have a direct 
bearing on the scientific success and value returned from the missions. 

The NESC Human Factors experts made three specific recommendations as a result of 
the ORR. These were incorporated into the ORR Board findings and recommendations 
that were submitted to JPL management. The JPL response to each is indicated. 

1 . Appoint an independent flight surgeon to evaluate on a regular basis the fatigue 
status of all team members. Although JPL elected not to recruit an “outside” flight 
surgeon, they did appoint the resident physician of JPL’s Safety Organization and 
Medical Services, Dr. Robert Estrada, to take on this role. Dr. Estrada effectively set 
up a comprehensive health and fatigue plan, including educational posters, visitation 
schedules and other prevention activities. 

2. Create alternative methods to get exhausted workers home. In response, JPL 
worked creatively to provide additional support to protect exhausted team members. 
Access to taxi’s or hotel rooms was made available and publicized to MER team 
members. 


9 

NASA Engineering and Safety Center 
Consultation Position Paper 

Document #: 

RP-04-10 

Version #: 

1.0 

Title: 

Mars Exploration Rover Flight Operations 
Technical Consultation 

Page #: 

7 of 47 


3. Establish maximum work limits and mandatory breaks from work. As 

discussed above, civil service and contractor operations personnel more or less 
followed the pre-defined work schedule, except during anomalous events. The 
scientific teams apparently did not follow such a prudent schedule. 

Flight Sciences (Entry, Descent and Landing) Discussion: 

The EDL Red Team was asked specifically to address three questions in regard to their 
oversight of the Project Reconstruction Team’s work: 

1 . Did they have the correct data and use it properly? 

2. Did they identify the correct issues and work them in the correct priority? 

3. Did they use the correct processes in doing their work? 

The NESC representatives provided answers to these questions that reflected the 
consensus of the Red Team as a whole. Briefly stated, the Project team used the available 
data very effectively and supplemented the data with ground testing of selected hardware. 
The team sought corroboration when possible, e.g. comparing accelerometer and gyro 
data or comparing IMU’s on rover and backshell. When no direct comparison was 
possible, they sought plausibility or consistency between different data sets. Key issues 
were identified promptly and appropriately prioritized. JPL assigned the highest priority 
within the Center to Spirit post-flight assessment and surface system operations. Key 
personnel, test hardware and facilities were made available to augment the EDL 
reconstruction process, including resources outside of JPL (e.g. at LaRC). There is no 
evidence that any important issues were overlooked. The overall reconstruction process 
however, was not well documented and thus was not completely clear to the members of 
the Red Team. In particular the process and schedule for making decisions regarding 
Opportunity’s up-coming EDL was not apparent. Nevertheless, the Project 
Reconstruction Team worked very well as a cohesive unit. The team members clearly 
understood what needed to be done and day-to-day activities progressed very well toward 
achieving the team’s objectives. Cooperation with the Red Team was excellent; the Red 
Team had timely access to available data and status of assessments. 

A number of technical issues arose regarding system performance of Spirit during EDL 
and regarding the adequacy of data available on the ground to inform the decision- 
making process about what steps should be taken for Opportunity’s EDL. A complete list 
is given on page 4 of the Red Team’s final report (Appendix F). Two issues of particular 
concern to the NESC representatives, identified during reconstruction and re-emphasized 
after both rovers had successfully landed, are discussed below. 
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1 . The reconstruction of the atmospheric density profile was based on accelerometer 
measurements and the assumption that the aerodynamic characteristics of the 
aeroshell were nominal. The reconstructed density was lower prior to parachute 
deployment and higher after parachute deployment than had been predicted on the 
basis of the atmospheric model used. Apparently, as a consequence of the initial 
low-density encounter, parachute deployment time, triggered at a specified 
dynamic pressure of 725 Pa, was later in time and at a lower altitude 
(approximately 2-sigma) than expected. Although this reduced the time margins 
to complete descent and landing to a low level, as measured by the parachute 
deployment altitude, margin was regained because the parachute descended more 
slowly than expected. The cause of this fortuitous “over-performance” of the 
parachute was not understood. Had relatively simple pressure and temperature 
sensors been included on the spacecraft for use during terminal descent and after 
landing, the physical situation actually encountered in situ during Spirit’s EDL 
would have been much clearer to the reconstruction team. The relative 
contributions of errors in the density profde model versus possible errors in drag 
coefficient could have been accurately assessed. As it was, the reconstruction 
team had to rely on assumptions and inferences in attempting to diagnose 
significant deviations from expected performance that had the potential for 
serious consequences for mission success. 

2. The aerodynamic performance of the entry vehicle did not conform to 
expectations from 175 thru 251 seconds after entry, just prior to parachute 
deployment. The angle-of-attack, while still well within safe margins, was higher 
than expected at flight conditions below Mach 6, with large-amplitude 
oscillations. Prior to entry there was an expectation of a dynamic instability when 
the vehicle decelerated to around Mach 3. The entry data showed that potentially 
this instability occurred as high as Mach 6. An instability occurring at a higher 
Mach number has a longer time to grow and hence can cause greater dispersions 
in attitude. The aerodynamic model used may be inadequate or incorrect, in the 
regime between Mach 6 and approximately Mach 3. The NESC representatives 
have emphasized that it is important for future planetary exploration missions to 
determine the cause of such instabilities. We are assured by the MER EDL Chief 
Engineer at JPL that this issue is under investigation at JPL. We do recommend 
that the results of the JPL study be published for the broader community to 
review, such as in an AIAA paper. 


As a result of the Spirit EDL reconstruction activities, both the JPL reconstruction team 
and the EDL Red Team made the same recommendations to JPL management regarding 
modifications of Opportunity’s EDL profile. These included a recommendation to not 
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change the entry flight path angle, but to increase the dynamic pressure at parachute 
deployment by 25 Pa as to increase the time margins on the parachute system. 

The NESC representatives and the Red Team as a whole recommended several areas of 
potential process improvement for future flights with similar EDL’s occurring in a short 
period of time. A software tool should be developed to visually display the EDL timeline. 
Multiple variables could be plotted and displayed simultaneously as a function of 
trajectory variables. In this way a more comprehensive and integrated view of the EDL 
sequence could be communicated. Secondly, a capability for automated frequency 
decomposition of the data would also improve the process. Frequency analysis by “eye- 
balling” may be sufficient for dominant modes, but it does not reveal power in other parts 
of the spectrum. Stacking up multiple data traces, denoting EDL events, and showing 
frequency content would be a powerful analytical approach. 

Conclusions: The MER expeditions have been outstanding successes, significantly 
exceeding original expectations both in terms of technical performance and scientific 
return. The MER Project at JPL and its broad support network performed with a very 
high level of dedication and competency. The Project and JPL are to be commended for 
their exemplary work and wonderful achievements. The broad national span of the NESC 
allowed us to bring together experts of the highest caliber from other NASA centers and 
from the U.S. Air Force to provide assistance and oversight in specific technical and 
scientific disciplines to the MER Project. 
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F. EDL reconstruction Red Team final report 
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APPENDIX A 


Biographical Sketches 
Cynthia Null: 

Before being detailed to the NASA Engineering and Safety Center as a Discipline Chief 
Engineer for Human Factors, Dr. Cynthia H. Null was a scientist in the Human Factors 
Division and Deputy Program Manager of the Space Human Factors Engineering Project. 
She began her career at NASA Ames in 1991 as a branch chief in the Human Factors 
Research Division. In 1996, she edited the Space Human Factor Requirements Definition 
for the Advance Human Support Technology Program. In 1997 while a program manager 
in the Aeronautics Enterprise, Dr. Null was part of the lead team designing the Aviation 
Safety Program and was the Ames Deputy Manager to the AvSP Program Office at 
Langley from 1997 until 1999. She was the acting Chief of the Human Factors Research 
Division from Spring of 1998 until she returned to research in late 1999. She teaches a 
course in Human Center Design for Aerospace Engineers at Stanford University. 

Dr. Null received a BA in Mathematics from Albion College, and an MA and Ph.D. in 
Quantitative Psychology from Michigan State University. Her career began with an 
academic appointment at the College of William and Mary, where she was on the faculty 
for 18 years before joining NASA. She has been the managing editor of the journal 
Psychometrika since 1984. 

John Caldwell: 


John Caldwell obtained his Master's degree in Experimental Psychology in 1979, his 
Ph.D. in Experimental Psychology in 1984, and his Pilot's Certificate in 1985. During his 
16 years with the U.S. Army's Medical Research and Materiel Command, he served as 
Chief of the Crew Stress and Workload Branch for 5 years before spending 10 years as 
the lead scientist over Aviation Sustained-Operations research at the U.S. Army's 
Aeromedical Research Laboratory. He has conducted numerous aviation performance 
studies in specially-instrumented simulators and aircraft. He has been a senior consultant 
to the Fatigue Countermeasures Group at the NASA Ames Research Center. Presently, 
and he is currently the Principal Research Psychologist for the U.S. Air Force's 
Warfighter Fatigue Countermeasures Program. He has published numerous peer- 
reviewed articles in scientific journals such as Psychopharmacology, Aviation, Space, 
and Environmental Medicine, Military Psychology, and Aviation Psychology, as well as 
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"user-focused" papers in operationally-oriented periodicals such as United Airlines' 
Safetyliner, the Airline Pilots Association International’s Centerline, Delta Airline's 
FlightLine, the U.S. Air Force Air Combat Command's Combat Edge, the U.S. Air Force 
Safety Center's Flying Safety, the German Forces Flight Safety publication 
Wehrmedizinishe Monatsschrift, and the U.S. Army's Army Aviation. He has authored 
book chapters on The use of stimulants to counter sleep deprivation and Sleep problems 
in aviation personnel, and he recently coauthored a book entitled Fatigue in Aviation: A 
Guide to Staying Awake at the Stick. He is an internationally-recognized expert in fatigue 
management in aviation continuous and sustained operations, and he is a member of the 
Speakers Bureau for the National Sleep Foundation's Drive Alert, Arrive Alive campaign. 
He frequently presents papers and provides fatigue-management workshops at scientific 
conferences and elsewhere. In addition, he offers instructional support for flight surgeons, 
commanders, and unit aviation personnel. 


Claude Graves: 


Mr. Graves is an internationally recognized expert in 
atmospheric entry design, analysis, and guidance. He is largely responsible 
for Shuttle entry trajectory and guidance design, as well as intimately 
familiar with Apollo, Gemini, and several planetary entry flight designs. 

In addition, he currently is involved as a lead at the Johnson Space Center for entry, descent, 
landing, and/or aerocapture for multiple planetary projects, such as the '07 
Mars Phoenix Lander and the '09 Mars Science Laboratory. 


Dean Kontinos: 


Dr. Kontinos’ career has focused on the development and application of 
computational methods for the design of thermal protection systems for hypersonic 
vehicles. He has developed computational methods for simulating fluid flow and 
heat transfer. He has also been active in developing procedures for incorporating 
high fidelity computations in the design process. He currently is Chief of the 
Reacting Flow Environments Branch at the Ames Research Center, where he is 
managing activities in computational analysis, entry vehicle design, and high 
enthalpy ground testing. Dr. Kontinos received his Ph.D. in Aerospace Engineering 
from North Carolina State University, Raleigh in 1994. 
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APPENDIX B 


Mars Exploration Rover Operations Readiness Review 
3-5 December 2003 

Human Factors Risk Assessment and Recommendations 

Cynthia H. Null, PhD 
NESC Human Factors Discipline Expert 


The two major categories of human performance that apply to understanding of risk to 
this mission are human fatigue and countermeasures, and operational procedures and 
interfaces. 

1. Human Fatigue and Countermeasures 

The effects of Mars’ time on the staff have been recognized by the MER leadership. 

They have been proactive, working with the Fatigue and Countermeasures Team from 
Ames Research Center. The project has taken many actions to protect their staff. These 
actions include: optional training in fatigue and countermeasures, staffing plans, nap 
room, room darkening shades in work areas, easy access to food during night shifts, and 
Mars local time watches. The project has directed the team leads to protect their staff by 
watching for signs of fatigue and recommending when a fatigue countermeasure (such as 
a nap, or ending a shift early) needs to be used. In addition, I believe that arrangements 
have been made to offer transportation home for overly tried staff. The project leadership 
has noted the time critical aspects of Entry-Descent-and-Landing phase (EDL) and the 
Impact-to-Egress. It is recognized that the operations of two vehicles simultaneously will 
add pressure and workload to certain staff. 

Other observations: There are no maximum duty standards or procedures to identify how 
much work is “too much work” or an objective measure of fatigue used by the project. 
Although the staff plan for this project after EDL for most operations entails a 4-days- 
on/3-days-off schedule, already several people are scheduled to work 10 consecutive 
days, 10-12 hour shifts. 

There is a belief, based on the “Pathfinder” experience, that surface operations will have 
decreased workload and will be less stressful than development. However, the program 
readily admits that they are not ready to operate the rovers and the science instruments. 

At the last PORT, the sequence development was not successful 2 out of 5 days. This 
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means that development of surface mobility and science operations tools will be 
proceeding during operations by the same team of people. As long as the tools are 
immature, it is not rational to believe workload will decrease. 

Evaluation of fatigue issues is left to the subjective judgment of the team leads or for the 
person to recognize that they personally are dangerously overworked. The team leads 
may have difficulty making this judgment especially when there is schedule pressure— 
when the people are needed to achieve for mission success. Both human limitations in 
self- evaluation and the needs of the mission, will limit a person from making the “I am 
overworked” judgment in a timely manner. Additionally, it is not clear who is looking 
after the team leads. 

The number of tasks that need to be accomplished by all teams in the next 3 weeks 
appears to be extensive, so the planned break before the EDL and ITE may evaporate. If 
the break disappears, the project needs to be especially attentive to human error during 
procedure execution. 

Although there was much sensitivity expressed that personal requirements will arrive at 
inconvenient times, I heard of no provisions to aid the JPL staff. 


Recommendations : 

A. Set objective criterion, after decisions with the ARC Fatigue and Countermeasures 
team, for maximum hours of work before a required break or an alternative objective 
measure of fatigue. 

B. Appoint an independent flight surgeon to evaluate the fatigue and health status of all 
project staff on an ongoing basis. There are two important criteria for this person: (1) 
experience in operational evaluations of fatigue and (2) someone the staff can trust. I 
believe that a military flight surgeon experienced in dealing with fatigue issues during 
long field operations could be a great help. 

C. Appoint a concierge to handle personal requirements that arise in conflict with 
mission schedule, to reduce the stress on the JPL staff. Note, such a person has been 
identified to assist the university scientists. 
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2. Operational Procedures and Interfaces. 

The timing and the structure of the review did not allow for evaluation of this important 
aspect of operations. For surface operations, the team is relying on the belief that the 
rover can not be harmed by inappropriate sequences — that is, its fail-safe features will 
protect the rover. I did not see a proof of this fail-safe capability — which is a concern. If 
this assumption is correct, the failure of procedures and interfaces could impact science 
data return but will not jeopardize the mission. However, the proof seems to be not a 
proof but a good feeling that the rovers are safe because they rejected sequences. This 
only indicates that some of the time incorrect sequences will be recognized. First time 
event procedures need to be followed carefully and sequences verified before they are 
uploaded to the rovers. 

From looking at high stress, high workload operations in other domains, one would easily 
conclude that the most important time to be diligent about safety and mission anomalies 
is when there is no time. Do not allow mission schedule and operational pressures to 
reduce normal reporting of incidents, unexpected outcomes, and off-nominal events, such 
as ISAs. When tasks are most complicated and schedules tight, and there is no time to 
document issues, the most important thing to do is document. 

Lessons learned: 

From other domains we can conclude the including human factors expertise during the 
development of operational procedures and interfaces can reduce workload and enhance 
the quality and error tolerant nature of the procedures. 

Evaluation of procedures and interfaces can be done during a PORT, but it is difficult or 
impossible to do in a formal review, as an ORR. 

Recommendation: 

D. Do not reduce requirements for documentation of incidents, unexpected 

outcomes, lessons learned and off-nominal events. Provide support to teams, for 
example an extra person without mission operations responsibilities to be 
responsible for such documentation. Make sure information gets provided to the 
teams operating Spirit and Opportunity. 
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Notes From Follow-Up Human Factors Visit To JPL/MER 

Cynthia Null 
March 6, 2004 


Frank Mortelliti arranged my visit to the MER team. 

Robert R. Estrada, Medical team lead, began the day with an upbeat briefing on how his 
office was working with the MER program to mitigate the risk that human fatigue can 
have on mission success though prevention and training. Prior to the MER ORR, the 
Ames Fatigue Countermeasures Group provided workshops for MER managers and team 
member on fatigue and sleep management. The JPL EAP published a resource guide for 
team members and their families. Posters on the recognition and management of fatigue 
were posted throughout MER work areas. 


Dr. Estrada described visits by the doctors that take place on Thursday nights and Friday 
mornings. I believe every team is visited approximately two times in every three weeks. 
The medical visits are informal - walking around and being available. Safety does a 
separate walk-around. I did not get the idea that either the doctors or safety would be 
unwilling just to say everyone needs a day off — let’s just skip the next SOL. 


It was not clear if any MER team members had taken advantage of nap rooms, or hotel 
rooms, or cabs home. 

Richard Cook, MER Program Manager, joined our discussions. He appeared to be fluent 
in sleep/duty concerns. 

There have been three events that have obviously put a lot of pressures on certain teams. 

1. Evaluation of EDL between Spirit and Opportunity landings, with the Red team — 
where margins were not understood 

2. Spirit airbag not retracting, requiring extra days to get off the landing craft, and 
the need to do a pirouette and alternate route exit — something they tried very hard to 
avoid. 

3. Spirit flash memory and rebooting problem — Cook said is that this was not a 
major technical problem, but the press put a lot of pressure on the team. ’’They all knew 
they would solve this quickly.” 

It appears that no special effort was made to check up on the teams during these times, 
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when fatigue risk was very high. I was given the impression that everyone switched to 
“right stuff’ mode and worked until issues were resolved. One of the issues brought up in 
general discussions at the ORR was the fact that for most technical positions there was 
little depth on the teams — that is only one or two people with skills, knowledge and/or 
training. This lack of team depth (redundancy) will be most apparent during off-nominal 
events. 

From Frank Mortelliti, “You’ll recall, the worst phase was ITE for Spirit when a lot of 
intense problem-solving required long hours of work. Since then, shifts have shortened to 
8-10 hours and, as Henry Stone says, ‘we’re getting better at what we do. ..it takes less 
time now.’ “ 

More recently, I would guess that the Opportunity Science team was worked to the bone, 
to prepare for the March 2 press conference. Leaks were happening, the science team did 
not want to be scooped, they needed to get everything lined up, and part of the team 
needed to get to the East Coast. At least two of the people I saw on TV on noon on 
Tuesday PST from DC, were seen at the lab before 2 pm Wednesday. Hopefully they had 
slept the entire weekend before their “15 minutes of fame”. 

After the informal discussions with Dr. Estrada, Richard Cook, and Frank Mortelliti, I 
was allowed to talk privately with anyone who was available, though there was some 
pressure to keep moving. I talked to 4 persons involved in sequencing and mission 
control. They all had slightly different stories to tell — not surprising. One person 
described how management had been helpful in adjusting work schedules to 
accommodate a family emergency. MER team members also admitted that in the 
beginning they were coming in on their days off, so they wouldn’t miss anything. They 
were all aware of fatigue and sleep management issues. 

I spent most of my time visiting with the Spirit Science team — their work schedule fit 
best into the timing of my visit. I saw science subteams that appeared to be working well 
together. The individuals I talked to, in general, admitted to being exhausted and 
unwilling to leave. I believe from a staffing view the science teams had little 
redundancy, so everyone needed (or felt the need) to work as much as possible. Even 
those who were apparently not exhausted said that the science teams did not work the 4 
days on, 3 days off schedules of operations. One scientist said that in the beginning he 
worked science for both rovers — since science teams work overnight SOLs and the 
rovers are 12 hours apart — this sounds like 24 hours to me. I talked to others who 
worked sequencing or up/down-link on their four days on, and then worked science on 
their three days off — this will involve a time shift of 8-10 hours at the beginning and end 
of each “3 days off’ and results in no time off. I did see cots in offices that look used. 
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I discussed work schedules for scientists with Safety, Dr. Estrada, and Richard Cook. 

They all agreed that the scientists were working whatever hours they wished. They said 
that this was individual choice — like not wearing a bicycle helmet (my analogy). They 
stated that this could be the only time in their career for these scientists to get this type of 
data, therefore this was all that mattered to the scientists and working was their choice. At 
least one person said, they [the scientists] are not in mission critical positions (I took that 
to mean since they were not controlling the rovers directly so they could not harm the 
missions). 

In our discussions of the possibility of setting duty limits, I was told that this 
could/would only be for critical personnel. I assume that referred to mission managers, 
since they make the key decisions. Most of these people work Earth time. It was stated 
that the limits needed to be sensitive to the mission design — not just the science of sleep 
and circadian rhythms. It seems that the MER management team believes they have 
enough safeguards on the rovers so that incorrect instructions to the rover will be rejected 
by the rover. That is, they are operating under the belief that operations could NOT flip 
over a rover or run a rover off a cliff. Everyone was reluctant to even discuss duty limits 
(although by setting work schedule to 10-12 hours 4 days on, 3 days off — they have 
essentially done this.) 

During my visit, I just saw tired scientists who were unwilling to go home, or even take a 
nap. I suppose they needed to make sure the data they cared most about would be 
collected on the next SOL. I saw people that seemed to be just fine — looked like normal 
scientists during a normal day — intense, animated, involved, etc. But even if scientists 
can’t jeopardize the mission, long- term sleep loss has a variety of very negative effects 
including reduced immune function, short temper, lack of judgment, lack of self- 
awareness and so forth. 

I asked for some data on use of hotels and taxis by JPL staff. 

When I was at JPL, the MER team was looking toward an extended mission. The intention 
would be to work two approximately 10 hour shifts synchronized to Earth time, which 
would make things easier on sequencing and mission control teams. They do not have the 
final plan worked out. I also have no idea how the science teams would handle this. It 
would mean robotic tasks and data approximately every other day, although I don’t quite 
understand how that works, with uplink/downlinks and Mars daylight. They have asked 
Ames Fatigue group to help with this re-planning. 

The MER team has taken a variety of proactive steps to assist team members in working 
Mars time. Dr. Estrada and this team have stepped in to provide a variety of services. In a 
main, it appears that these activities have been successful for the mission operations and 
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sequencing teams. I remain concerned that the scientists are ignoring much of the advice 
provided on managing fatigue and sleep, and this has been accepted by JPL. 
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APPENDIX C 

Mars Exploration Rover Operations Readiness Review 
Workload Fatigue Risk Assessment and Recommendations 

John Caldwell, Ph.D. 

NASA Engineering and Safety Center/U.S. Air Force Research Laboratory 

1. On 3-5 December 2003, a Mars Exploration Rover (MER) Operations Readiness 
Review (ORR) was conduced in the Building 167 Conference Room at Jet Propulsion 
Laboratory (JEP), Pasedena, CA. As a Review Board member representing the NASA 
Engineering and Safety Center, the following observations/recommendations are offered 
based upon the material presented by MER personnel: 

a. Observations on the positive side : Although the majority of the briefing 
content was focused more on “engineering issues” rather than “human performance 
concerns,” it was clear that issues of personnel workload and the fatigue from shift work 
and extended duty periods were of importance to the MER staff. Most noticeably, the 
project leadership has noted that the time-critical work to be accomplished prior to the 
rapidly-approaching Entry-Descent-and-Landing phase (EDL) and the Impact-to-Egress 
phase along with the probable (but as yet unknown) requirements to handle unexpected 
events, etc. will likely place a great deal of strain on the teams responsible for ensuring 
success in these areas. In addition, there was recognition of the fact that the workload 
required for handling Surface Operations on the currently-planned schedule for a 3- 
month period certainly would be daunting given current staffing levels. Furthermore, 
MER personnel had given thought to the circadian-adaptation problems posed by the 
necessity to continuously rotate work schedules by 40 minutes per day in order to 
accomplish tasks on a time schedule corresponding to the Mars day (as opposed to the 
Earth day). The MER leadership should be commended for considering these issues and 
for working with members of the Fatigue Countermeasures Group at NASA Ames 
Research Center to understand and address many of the known fatigue-related concerns. 
Specifically, it was noted that the following provisions have been made: 

1) . In light of the fact that 10-12 hour work shifts have been deemed necessary, 
the overall staffing plan for most operations entails a 4-days-on/3-days-off schedule to 
attenuate a dangerous escalation in cumulative crew fatigue throughout the mission (this 
is particularly the case for Surface Operations). 

2) . To help mitigate excessive fatigue in situations where personnel must remain 
at JPL for excessively-long duty periods, napping facilities have been provided on site. 

3) . To prevent overly-fatigued personnel from placing themselves at risk for a 
drowsy-driving mishap, arrangements are in place to provide sleeping quarters on site or 
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at a nearby hotel. In addition, I believe there was a provision to offer transportation home 
for fatigued individuals. 

4) . JPL has ensured that necessary services (i.e., food, medical, etc.) will be 
provided to MER personnel despite the fact that such services will be required at odd 
times according to a typical Earth-day schedule. 

5) . Asa general preventative measure, the MER staff provided training to 
managers and staff members on the causes of operator fatigue, the potential consequences 
of improperly-managed fatigue, and the scientifically- valid strategies for countering 
fatigue during the course of the mission. 

6) . In very general terms, the Ames Fatigue Countermeasures Group was 
apparently consulted on crew scheduling issues, although it appears that they were not 
fully involved in the development of optimal staff schedules. 

7) . For the science teams, JPL has arranged for off-center housing that will help to 
facilitate operator sleep (and subsequent alertness) during MER shift-work operations. 
Examples of special considerations include: a) securing “blocks” of apartments so that 
the shift workers will be living in relative isolation from sleep-disrupting factors, b) 
making arrangements for housekeeping services to be performed on a shift- work- friendly 
schedule, and c) preparing rooms outfitted with blackout curtains. 

b. Observations on the negative side : Despite the recognition that human fatigue 
will almost certainly be a consideration during every phase of the MER mission, most of 
the MER leadership seems to be underestimating its potential impact on crew safety and 
effectiveness. In addition, the leadership is, in my opinion, placing far too much faith in 
their own ability to monitor the fatigue levels of staff members and to accurately gauge 
the extent to which staff fatigue is threatening day-to-day mission readiness. The 
following specific problems were noted: 

1) . Despite the existence of staffing planes, there appear to be no “hard-and-fast 
rules” about maximum working periods (hours per day) or maximum consecutive duty 
periods (days per week or month) beyond which personnel simply will not be allowed to 
work. 

2) . Since there are no stated maximums, there also are no formalized procedures 
for identifying how much work is “too much work” for any given employee or group, and 
how (and whether) an unavoidable increase in workload requirement will be safely 
handled. 

3) . Although fatigue training was offered for all MER staff members, this training 
was apparently made optional despite the fact that improperly-managed fatigue is a 
known threat to operational safety and effectiveness (and has in-fact been identified as a 
causative or contributing factor in several very high-profile disasters such as the 
Challenger mishap for instance). 

4) . There is no mechanism or provision for an on-going “fatigue assessment” of 
MER personnel beyond simply relying upon the subjective judgments of the team leaders 
(who may end up being overly-fatigued themselves, and/or who may be susceptible to 
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unintentionally ignoring the presence of undesirable fatigue levels in favor of maintaining 
a high staffing profile). 

c. Recommendations '. At this late stage of the game, it may not be feasible to 
develop and validate formal duty-limitation plans and procedures. Also, there probably is 
insufficient time to formulate a workable set of procedure that can be implemented to 
temporarily safely exceed the published limitations (given that specific fatigue 
countermeasures are implemented). Both of these tasks would require extensive 
coordination among MER leaders and the fatigue experts at NASA Ames (or elsewhere). 
Finally, it may also be too late to schedule make-up fatigue training sessions for 
managers who elected not to attend the earlier sessions provided by Ames personnel. 
However, there are a couple of recommendations that would immediately provide a 
substantial reduction in the fatigue risk: 

1). An operationally-experienced NASA or Air Force (or Army) flight surgeon 
should be immediately assigned to monitor all phases of MER operations, and to 
perform face-to-face subjective fatigue assessments of staff members on a routine 
basis (i.e., several times per week). Care should be taken to recruit/assign a senior- 
level flight surgeon who has had considerable experience working within the context 
of demanding NASA space missions (in the case that a NASA flight surgeon is 
chosen) or within the context of demanding military missions in which sustained 
operations were required (in the case that an Army or Air Force flight surgeon is 
chosen). This flight surgeon should be completely independent of the existing MER 
leadership, and if possible even independent of the onsite JPL leadership. His/her 
decisions about the status of individual staff members and actions to ensure the 
safety, alertness, and effectiveness of these staff members must be binding. It is 
highly recommended that the assigned flight surgeon be required to coordinate fully 
with an on-site fatigue expert from Ames (or the Air Force) since considerable 
expertise in this particular area is crucial. 

2). Since it is clear that there will be several relatively short periods of time (i.e. 
9-10 days) in which the operational pace will reach unusually high levels, and since an 
experienced flight surgeon will be on site to oversee various fatigue 
treatment/countermeasure options, it is strongly recommended that a policy allowing the 
use of pharmacological fatigue adjuncts be immediately developed and implemented. 

-Pharmacological adjuncts should include, at a minimum, sleep-inducing 
agents to maximize the restorative value of crew napping opportunities. The 
primary agents that should be considered for this purpose are zolpidem (Ambien) 
and zaleplon (Sonata), both of which have been shown to safely and reliably 
induce sleep without producing unwanted residual effects upon awakening. 

These agents will enable effective napping even when the naps are unavoidably 
placed at periods of time where restful sleep will be difficult to obtain. Research 
has shown that naps induced with these agents ultimately enhance subsequent 
performance due to improved sleep quality. 
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-Pharmacological sleep-inducing agents also should be available to help 
promote and maintain the daytime sleep of MER shift workers, especially for the 
first 2-3 days of a shift rotation. One excellent agent for this purpose is 
temazepam (Restoril) which has been shown to enhance the quality of 6- to 8- 
hour non-standard sleep periods. The use of temazepam as a daytime sleep 
enhancer has been shown to enhance subsequent performance. 

-For limited periods of time during which nap/sleep opportunities are 
virtually nonexistent due to extremely high work demands, some consideration 
should be given to developing a policy and plan permitting the short-term use of 
alertness-promoting compounds. Of course, caffeine in the form of beverages, 
candies, and/or caffeinated gum should be considered first. However, modafinil 
(Provigil) also should be made available for flight surgeon prescription on a 
limited and controlled basis. Modafinil was recently FDA approved for the 
treatment of on-the-job sleepiness associated with shift work — a problem that will 
certainly be experienced by many members of MER operations. Modafinil also 
was just approved for use in certain types of Air Force flight operations. Both 
caffeine and modafinil have been proven safe and effective by military and other 
research organizations. 

Such adjuncts should not be used as a replacement for well-planned and well-executed 
work/rest schedules. They also should not be utilized until after non-pharmacological 
fatigue countermeasures have been exhausted. 

3). In order to reinforce the fatigue-countermeasures training that has already 
been offered and to provide an opportunity for MER staff members to address specific 
problems that arise as a function of the ongoing mission schedule, brief refresher courses 
in fatigue countermeasures should be offered. Short refresher courses should be planned 
at least at the midpoint of surface operations, and the courses should be offered at a 
variety of times so as to minimize interference with the work schedule. These courses 
should be conducted by the NASA Ames Fatigue Countermeasures Group and/or the Air 
Force Research Laboratory (AFRL) Fatigue Countermeasures Branch in coordination 
with Ames. 

2. In summary, it appears that the MER project already has made significant 
strides towards proactively addressing the risks associated with the fatigue that will no 
doubt stem from extended duty periods and circadian misalignment. The MER team 
should be commended for their considerable efforts in this regard. However, there 
remains a degree of complacency regarding the insidious effects of fatigue that should be 
aggressively attacked. To further mitigate the likelihood that crew fatigue will adversely 
impact MER operations (all phases), continuous on-site flight- surgeon support should be 
immediately arranged, flight-surgeon decisions should be made in concert with experts 
from NASA Ames, a pharmacological fatigue-mitigation plan should be developed and 
implemented for acute high-workload and transitional periods, and refresher fatigue- 
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countermeasures training should be scheduled at least at the midpoint of surface 
operations. 
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APPENDIX D 


MER RED TEAM REPORT 1 
January 6, 2004 
Claude Graves 


General Comments: 

• Overall the Mars Approach and the Spirit Entry, Descent, and Landing (EDL) for 
the Spirit spacecraft was very successful with the EDL systems performing well. 

• Based on a recommendation of the EDL Red Team during PORT 10 in 
December, 2003 the EDL reconstruction team implemented a chart that summarizes 
the key EDL events along with the limiting values, expected values, and estimated 
actual values. This was used during the remainder of the PORT 10 simulation and 
is being used by the Spirit reconstruction team. This has improved the 
reconstruction process and has been very useful tool to improve the help coordinate 
the reconstruction process and to provide visibility into the reconstruction status. 

• The reconstruction team is developing a plan for completion of the remaining 
work. This will be helpful in understanding the overall reconstruction status. 

• The reconstruction is incomplete with very little, if any, high frequency data 
received at this time so the reconstruction is preliminary and with some changes 
possible as the reconstruction progresses. 

• The reconstruction team will begin assessment of the changes needed, if any, for 
the Opportunity EDL. 


Specific Comments: 

• The approach trajectory control was very accurate resulting in deletion of the last 
planned Trajectory Control Maneuver (TCM) and the later contingency TCM 
opportunity. The entry flight path angle is estimated to be -1 1 .494 plus or minus 
0.01 deg compared to a target value of -1 1.5 plus or minus 0.12 deg 3 sigma. 

• The key entry parameters are presently reported to be near expected values. 

• The atmospheric density profile deviated from the expected density profile with 
the reconstructed density profile estimated to be about 85 % of the expected density 
at about 25 km altitude. This density deviation was lower at bother higher and 
lower altitudes within the sensible atmosphere. The atmosphere experts have not 
endorsed this reconstructed profile The reconstructed density profile is based on 
accelerometer measurements and an assumption that the aerodynamic 
characteristics are nominal. 

• The parachute system was deployed based on an in-flight computation that 
ensures that the parachute is not deployed until the dynamic pressure has reduced to 
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an acceptable value. This deployment logic worked very well and resulted in the 
parachute being deployed very near the nominal dynamic pressure with an actual 
deployment at 729 Pa vs. planned 725 Pa. The deployment time was later than 
expected was probably caused by the atmospheric variations discussed earlier. The 
resulting peak aerodynamic load on the parachute was estimated to be 11725 vs. 
13880 planned. 

• The parachute inflation time appear to be about as expected with an estimated 
inflation time of 1.33 to 1.58 sec compared to a nominal time of 1.3 sec. 

• The delayed parachute deployment reduced the time margins to complete the 
descent and landing, as measured by the parachute deployment altitude, to a low 
level. However, slower descent while on the parachute resulted in comfortable 
margins for completing the descent and landing. The cause of this slower descent 
while on the parachutes is not fully understood at this time. 

• The system that estimates the spacecraft translation speed relative to the Mars 
surface while on the parachute successfully determined the relative speed. This 
information was used to compute the propulsive maneuver needed to reduce the 
translation speed to protect the air bag system during the landing. The resulting 
horizontal relative speed at bridle cut was about 1 1 .5 m/s compared to a limit of 2 1 
m/s. 

• The system to compute the horizontal relative speed uses photographs taken in 
sequence to estimate the relative speed. One of the photograph comparisons was 
deleted because the filter limits on the crater ridge definition was exceeded. 
However, the relative speed was successfully computed using the remaining 
information. 

• The bridle cut altitude was lower than expected and the altitude rate at bridle cut 
was higher than expected. 

• The landing loads were lower than the allowable, 8 to 10 g’s vs. a limit of 40 g’s. 

• The landing point is estimated to be about 9 km from the target and about 12 km 
from the pre EDL estimated mean landing point. 

Planning for Opportunity: 

• There are three items to be investigated to see if changes should be made for 
Opportunity: 

o Increase the time margins for terminal descent 

• Reduce entry flight path angle 

• Increase the dynamic pressure at parachute deployment 

• Change the post parachute deployment time line 
o Increase bridle cut altitude 

o Change the Dimes filter parameters to allow for a less clear definition of 
crater ridge 
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MER RED TEAM REPORT 2 
January 16, 2004 
Claude Graves 


Specific Comments: 

One addition needs to be made to the specific comments about the Spirit EDL 
reconstruction and assessment defined in the MER RED TEAM REPORT by Claude 
Graves on January 6, 2004. The angle-of-attack was outside the expected bounds for 
speeds below about Mach 6 with large amplitude oscillations, but was within acceptable 
limits by a considerable margin. Potential causes of exceeding the expected bounds of 
the angle-of-attack were identified but the specific cause was not isolated. Initial 
examining the effect of retargeting to a shallower entry flight path angle using the 
potential causes of the large angle-of-attack showed a small degradation of the angle-of- 
attack envelope. This was one reason for not retargeting the entry flight path angle to 
increase the time on the parachutes. 

Changes for Opportunity: 

The Red Team was asked about the changes recommended for Opportunity EDL and the 
team recommended not changing the entry flight path angle because of the Spirit EDL. 
There could be subsequent changes to the entry flight path angle targeting if weather 
conditions along the entry flight path change based on real time atmospheric 
measurements. The Red Team did reco mm end increasing the dynamic pressure at 
parachute deployment by 25 Pa. to increase the time margins on the parachute system. 

The EDL reconstruction and assessment team made the same recommendations to the 
MER Project management and these recommendations were taken to the JPL 
management. 

It is interesting that the Red Team was asked to make a recommendation for Opportunity 
changes before the EDL team was asked. I do not believe this influenced the EDL team 
recommendation, but both groups concurred on the recommendations. 


MER RED TEAM REPORT 3 
January 23, 2004 
Claude Graves 


Recommendations for improvement 

The following recommendations for improvement are recommended for future flights 
with similar EDL’s occurring in a short period of time. In some cases the 
recommendations may apply to independent missions that include EDL. 
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• Include sufficient time and resources in the project plan to allow for development and 
demonstration of a rapid capability to complete the needed trajectory and systems 
reconstruction and assessment to affect the later EDL. 

• Include pressure and temperature sensors on the spacecraft for use during terminal 
descent and after landing. This information is necessary to be able to differentiate 
between atmospheric density and drag coefficient effects. This applies for all future 
missions that include EDL and do not have an independent method for acquiring this 
information.. 

• Enhance the display of flight data to allow use of multiple independent variables such 
as time, energy, and speed and include the ability to superimpose mission events. 


MER Spirit EDL Reconstruction and Development Assessment 

January 12, 2004 
Claude Graves 


EDL Team Assessment 
General Comments: 

The EDL reconstruction and development team is well qualified, motivated, and has an 
appropriate set of tools to accomplish their objectives. The activities are much better 
coordinated than during PORT 10. I was impressed with the quality and quantity of work 
as well as the cooperation among the team members and with the EDL RED Team. 

Effective use of data: 

The team has utilized the available data very effectively and has some conducted ground 
test of selected hardware to supplement the flight data to help understand some of the 
concerns. The data was properly assessed and provided to support the decision process 
for MER B. 

Identification of the correct issues and working in the correct priority: 

As expected the schedule is very tight and data availability was affected by data 
transmission needed to operate the surface systems. The team has promptly identified the 
correct key issues and appropriately prioritized the work effort to best use the resources 
in preparation for a timely decision for any changes needed for MER B. JPL assigned the 
post flight assessment and surface system operations of the MER A the highest priority 
within the Center and made available key personnel and test hardware and facilities to 
augment the EDL reconstruction and development. 
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The correct process: 

While the EDL Red Team would have preferred to have better insight into the overall 
process at the beginning of this effort the day-to-day activities progressed very well 
toward achieving the team objectives. Daily assignments were clear and effective with 
appropriate groups being formed for issues that involve multiple work areas. The EDL 
Red Team was accepted by the EDL reconstruction and development team as a partner in 
the process with the Red Team having timely access to available data and status of the 
assessments and with Red Team inputs included into the decision process. 

Decisions: 

The EDL reconstruction and development team made the correct recommendations to the 
management Teams. 


1 Flight Mechanics Team 

The Flight Mechanics team is also well qualified, motivated, and had an appropriate set 
of tools. The data was used properly with cross checks on the validity of the data where 
this is possible. The key issues were identified and the work was properly prioritized. 
Key resources at LaRC were effectively used to address identified issues. Although a 
process for this work was not documented the process worked smoothly with the correct 
interfaces and teaming arrangements developed to effectively complete the reconstruction 
and development activities. The team members were very cooperative and respectful 
within the team and outside the team and appropriately incorporated the EDL Red Team 
it their activities. 
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APPENDIX E 

Report to NESC, 1/16/04 

MER EDL Red Team , 1/3/04-1/9/04 

Dean Kontinos 

This report documents my participation as a member of the Mars Entry Rover (MER) 
Entry-Descent-Landing (EDL) Red Team. The Red Team was commissioned by the JPL 
Deputy Center Director to act as an independent group evaluating the data and process 
used by the MER EDL team to reconstruct the MER-A EDL sequence, and potential 
modification to the MER-B entry. 

The Red Team was chaired by Glenn Cunningham, independent consultant and retired 
JPL manager, and comprised of Bobby Braun, Georgia Tech; Gentry Lee, JPL; Dankai 
Liu. JPL; Bob Mase, JPL; Carl Peterson, Sandia National Labs; Dave Spencer, JPL; Sam 
Thurman, JPL; Jeff Umland, JPL; and Gordon Wood, consultant. In addition were 2 
NESC representatives: myself and Claude Graves, JSC. I took the place of Charles Smith, 
ARC, who was an original member of the team. 

The Read Team was asked three specific questions by the JPL Deputy Center Director. In 
addition, the Red Team was asked by the MER Project Manager to state a position 
regarding modification to the MER-B flight path angle. 

I provided a written report to the Red Team Chairman containing my answers to the three 
questions. I also provided comments and suggestions on process enhancement. The 
entirety of my report to the Red Team Chairman is contained herein. 

In summary, the Red Team recommended that no change be made to the MER-B entry 
flight path angle. This recommendation was unanimous within the Red Team and 
consistent with the EDL Team recommendation. A follow-on recommendation was to 
increase the parachute deploy dynamic pressure by 25 Pa. There was unanimity within 
the Red Team for this follow-on recommendation. The parachute deploy dynamic 
pressure issue is still being investigated by the MER program. Details of the rationale of 
the Red Team recommendations are documented by the Chairman and can be provided to 
the NESC upon request. 
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MER EDL Red Team , 1/3/04-1/9/04 
Final Report, 1/12/04 
Dean Kontinos 

This report contains my observations and findings as member of the MER EDL Red 
Team. The review period is 1/3/04 to 1/9/04, which covers MER-A entry, MER-A 
reconstruction, and MER-B EDL TCM decision. 

Before answering the specific questions that defined the objective and scope of the Red 
Team, a few comments on the quality and character of the EDL Team is appropriate. In 
my view, the EDL team was exceptional in both the technical capacity of the individual 
members and their professional interaction as a group. As individuals, the members were 
extremely knowledgeable, diligent, and enthusiastic. As a group, they were respectful to 
each other, tolerant of opposing views, and allowed time for everybody to be heard. 

There was an expectation of excellence while at the same time a willingness to support 
each other. This group dynamic, whether natural or learned over time, seemed to be an 
innate part of their group dynamic since there was little intervention by leaders to enforce 
group etiquette. Rather, the group was self-policing. Equally important was that members 
of the group challenged each other. Members were sufficiently broad in technical depth 
to question the engineer in charge of another area. The atmosphere was such that one’s 
data and analysis was questioned, but not the individual’s technical authority. Finally, 
there was an extraordinary amount of work performed during the week by associated JPL 
and LaRC personnel. Every day there was presented an analysis or test performed by an 
unheralded engineering working through the night. 

Did they use the data properly? 

The EDL Team did a great job extracting as much information as possible from the 
available data. The team sought corroboration when possible, e.g. comparing 
accelerometer and gyro data or comparing IMUs on rover and backshell. When no direct 
comparison of engineering data was possible, they sought plausibility or consistency 
between different data sets. In other words, if an event was postulated based on one type 
of data, consistency was sought with a different type. There always was somebody to 
argue the counter-point to every proposed theory. This process mitigated excessive 
speculation or extrapolation. 


2 Did they identify the correct issues and work them in the correct priority? 

The issues they did identify were definitely important. The EDL timeline margin was 
paramount because of the impending proposed TCM. Team priorities were set based on 
relevance to the TCM decision. I was not able to identify other issues that weren’t 
covered by the team that I thought were a priority. Perhaps we all overlooked something. 
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3 Did they use the correct process in doing their work? 

I was a late edition to the Red Team, had not participated in the previous PORT, and 
therefore was minimally prepared for Red Team duties. Perhaps this lack of preparation 
was the primary reason that the EDL reconstruction process was not clear to me. It 
seemed like there was an unspoken, undocumented, unadvertised process in the collective 
heads of the team. The team all seemed on the same page, yet I was confused much of the 
time regarding the direction of the team. More specifically, I understood the individual 
activities, but the process and schedule for making a MER-B decision was not always 
apparent. The only firm conclusion I can draw is that the process was definitely not 
documented. 

Along the lines of the previous point, it might have been useful to step through the 
decision logic prior to final resolution of the data. In other words, “what-if ’ thinking 
regarding the outcome of particular analyses could have been performed. For example, 
what if there is or is not convergence between the reconstructed atmosphere and the 
atmosphere model? How would the opposite results of that analysis affect the decision 
process? This type of thinking would serve to focus the team on the key issues as well as 
highlight the key data that actually affect the decision. I’m not advocating endless 
hypothetical speculation, but a little forward thinking. 

A key issue facing the EDL team was a 2-sigma late parachute deploy time. After 
atmosphere reconstruction, it was stated that a combination of 1 -sigma dispersions on 
atmospheric density, winds, and drag coefficient are sufficient to cause a 2-sigma event 
in the chute deploy. This analysis is key because it supports the contention that the EDL 
systems behaved as they should, and that the unusual 2-sigma chute deploy event was the 
result of an unlikely combination of flight circumstances. The sufficiency of the 
combination of 1 -sigma circumstance to cause the 2-sigma event was stated, but no data 
or analysis was ever shown. I’m not doubting the veracity of the analyst, only noting that 
a presentation of the data was not given. 

If ever a back-to-back entry is to be executed in the future, I have 2 recommendations for 
process improvement. The first improvement is a development of a software tool to 
visually display the EDL timeline. Multiple variables could be plotted and displayed 
simultaneously as a function of trajectory variables. For example, it would have been 
useful to see the accelerometer data plotted as a function of time along with entry speed, 
Mach number and altitude. EDL events could also be noted on the timeline. A simple 
switch would then permit plotting of the variables as a function of altitude, for example. 

In this way, a more comprehensive and integrated view of the EDL sequence could be 
communicated. The second improvement is automated frequency decomposition of the 
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data. Several times I witnessed a frequency analysis by “eye-balling” the data. Although 
eyeballing is sufficient for the dominant modes, it does not reveal power in other parts of 
the spectrum. Stacking up multiple data traces, denoting EDL events, and showing 
frequency content would be a powerful analysis approach. 

The final observation regards the use of the Red Team itself. During the final meeting 
regarding the TCM decision, the project manager polled the Red Team for their opinion 
before the EDL team had unequivocally stated their recommendation. It seems in the end 
that the decision was not particularly controversial, nevertheless, the engineering team 
should be issuing their recommendation without influence from the Red Team. 


Report to NESC, Addendum, 3/26/04 
MER EDL Red Team , 1/3/04-1/9/04 
Dean Kontinos 

On 1/16/04, 1 had submitted a report to the NESC describing my participation on the 
MER EDL Red Team. The report contained some preliminary remarks describing the 
objectives of the Red Team, and then contained my submittal (in its entirety) to the Red 
Team Chairman Glenn Cunningham for his use in drafting the final team report. 

Since the completion of that activity, I have pursued two lines of follow-on inquiry, 
primarily based on my own interests and expertise. This addendum to my original report 
describes my follow-on activities. 

The first line of inquiry is in the unexpected aerodynamic performance of the entry 
vehicle. The specific issue as express in Red Team final report 1 is the following, 

• “Higher than expected attitude of lander (angle of attack) between 175 thru 25 1 
sec after entry (just prior to chute deploy) (ISA Z83007 ) ” 

This higher than expected angle of attack might be explainable by a mischaracterization 
of the dynamic stability of the entry vehicle. Very roughly, prior to entry there was an 
expectation of a dynamic instability when the vehicle decelerates to around Mach 3. The 
entry data shows that potentially this instability occurred as high as Mach 6. An 
instability occurring higher in Mach number means that instabilities have longer time to 
grow and hence cause greater dispersions in attitude. 

At Ames Research Center, I consulted with Dr. Ethiraj Venkatapathy on potential causes 
for high Mach number dynamic instability. We both thought it important for future 
exploration missions to determine the cause of such an instability. Furthermore, we 
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thought the NESC would be an appropriate organization to perform the analysis. Dr. 
Venkatapathy contacted Wayne Lee (JPL), MER EDL Chief Engineer, to discuss the 
issue. I was not present during their conversation, but was told by Dr. Venkatapathy that 
Wayne Lee and his team were still investigating this issue, and that Wayne felt JPL had 
sufficient and appropriate staff to complete the task. I concur that the EDL team is well 
equipped to address the issue. I do recommend that the results of their study be published 
for the broader community to review, such as an AIAA paper. 

The second follow-on inquiry was in regards to the Red Team recommendation, 

“Relative to atmospheric reconstruction, it is suggested that the Project was remiss in not 
flying pressure and temperature sensors for use during terminal descent and on the 
ground. Such sensors, with a likely mass of hundreds of grams, would have allowed 
separation of the density and drag coefficient errors sources, something for which the 
Reconstruction Team could only speculate. It is suggested that these sensors be added to 
the EDL reconstruction requirements of future landed systems.” 1 

I think it beneficial to future missions to understand the relationship between engineering 
instrumentation and accuracy in EDL reconstruction. A study could be performed by the 
NESC that derives instrumentation requirements for particular levels of accuracy in EDL 
reconstruction. 

I discussed this idea with Dr. Venkatapathy, and he subsequently discussed it with 
Wayne Lee. I am told by Dr. Venkatapathy that Wayne Lee has been asked a similar 
question by Dave Lavery, Program Executive for Solar System Exploration, NASA HQ, 
but specific to MER. I recommend that the NESC review the response given by Wayne 
Lee, and then determine if a broader analysis is required. It may be that in answering the 
question specific to MER, the answer is broadly applicable and no further analysis is 
warranted. 

In summary, since completion of my participation as NESC representative to the MER 
EDL Red Team, I have further inquired about the unexpected excursion in angle of 
attack, and the Red Team recommendation for additional flight instrumentation. At this 
time, the MER project is responsibly pursuing both issues. I do not see need for NESC 
participation at this time. I do recommend that the results of the aerodynamic 
investigation be published in at least a conference paper, and the instrumentation analysis 
be review by the NESC to determine the range of applicability. 
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[1] Final Report of the Mars Exploration Rover (MER) Entry Descent and Landing 
(EDL) Red Team, Glenn Cunningham (Chairman), Jan. 17, 2004 
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APPENDIX F 


Final Report of the 
Mars Exploration Rover (MER) 
Entry Descent and Landing (EDL) 
Red Team 


Glenn E. Cunningham 
Chairman 

January 17, 2004 


Jet Propulsion Laboratory 
California Institute of Technology 
Pasadena, California 
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Final Report of the 

Mars Exploration Rover (MER) 
Entry Descent and Landing (EDL) 
Red Team 


3.1.1 Executive Summary 

The MER EDL Reconstruction Red Team assessed the performance of the MER 
Project’s MER-A EDL reconstruction activities. It observed an extremely well qualified 
reconstruction team that performed in an exceptional manner. The team had extensive 
performance data and utilized it well. The team rapidly identified the salient issues and 
properly prioritized their analysis of them. While their processes were not well 
documented in the normal flight operations manner, their previous experience with the 
design of the EDL and their significant expertise enabled them to perform in the 
exceptional manner observed. The Red Team finds that the MER-A reconstruction was 
well executed, that the proper issues were analyzed, and proper recommendations were 
made relative to modifications for the MER-B EDL. Some suggestions for future 
reconstruction activities are offered. The members of the Red Team felt extremely 
honored to have been participants in this activity. 
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INTRODUCTION 

The Mars Exploration Rover Entry, Descent and Landing (EDL) Red Team was 
chartered by the JPL Associate Director for Flight Projects and Mission Success to assess 
actual MER-A (Spirit) EDL reconstruction activities and results to identify any 
weaknesses/anomalies in the MER-A EDL for the benefit of MER-B (Opportunity). The 
Red Team’s membership is shown in attachment 1. 

The Red Team assessed the performance of the MER EDL Reconstruction Team during 
PORT 10 and provided a previous report on those findings. 

This report deals with the performance of the Project’s MER-A EDL reconstruction 
activities. 


RED TEAM OPERATIONS 

The Red Team convened to witness the actual MER A EDL events on January 4, 2004, 
and then again each day from January 5, 2004 (sol 2) through January 8, 2004 (sol 6) to 
assess the EDL reconstruction activities in the Reconstruction Team’s war room. The 
Red Team assigned a member or members to each of the Project’s reconstruction sub- 
teams. (For assignments, please see attachment 2.) The Red Team members assessed 
each sub-team’s approach to the apparent issues and, in some cases, participated with the 
sub-team’s analysis, as requested. On a daily basis, the full team sat- in on the full 
Reconstruction Team tag-up meetings. 

A daily assessment was prepared, and was sometimes presented to the Project 
management. 

After sol 6, it was determined that the Reconstruction Team had identified the significant 
performance issues of the MER-A EDL and had established a satisfactory plan to 
disposition each issue with respect to the MER-B EDL. Thus, the Red Team chose to 
stand-down, however the Chairman continued to track the continuing activities of the 
Reconstruction Team. 

On a daily basis, the Red Team asked itself the following questions as tools to assess the 
Reconstruction Team’s activities: 
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• Did they have the correct data and use it properly? 

• Did they identify the correct issues and work them in the correct priority? 

• Did they use the correct processes in doing their work? 

The following section summaries the Red Team’s assessment of these questions. 


OVERALL PERFORMANCE OF THE RECONSTRUCTION ACTIVITY 

The EDL Reconstruction Team was well qualified, motivated, and had an appropriate set 
of tools to accomplish their objectives. The actual MER-A EDL activities were much 
better coordinated than the Red Team observed during PORT 10. We were impressed 
with the quality and quantity of work as well as the cooperation and exceptional strong 
team dynamics among the team members. There was an expectation of excellence while 
at the same time a willingness to support each other. The group was self-policing. 
Equally important was that members of the reconstruction team challenged each other. 
Members were sufficiently broad and had the technical depth to question the engineer in 
charge of another sub-team. The atmosphere was such that one's data and analysis was 
questioned, but not the individual's technical authority. Finally, there was an 
extraordinary amount of work performed during the week by associated JPL and LaRC 
personnel. Every day there were new analyses presented or tests results described from 
unheralded engineering work performed throughout the night. 

Relying on their extensive development experience and a tremendous amount of critical 
event reconstruction data (MGS relay, ODY relay and DTE playbacks), the MER EDL 
team did an outstanding job analyzing the MER-A entry, descent and landing system 
performance. Their process employed an appropriate balance of data discovery, 
interpretation and debate. This process was completed quicker than planned due to the 
exemplary performance of the MGS and Odyssey spacecraft in returning this critical data 
and a well-organized process of data distribution, led by Erik Bailey. 

The team was careful to use the data properly, defining which insights were gleaned 
directly from the data products and which were inferred. When questions arose outside 
the expertise of sub-team members, they immediately brought others into the group 
discussion. After a day of data discovery, they identified atmosphere reconstruction and 
chute-deploy timing as priority issues and began to work them with the proper priority. 

In the following days, as additional data became available, DRL performance and chute 
deploy attitude were added to this high priority list. The proper priorities were 
established. The team jumped on any parameter that was observed to violate its expected 
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value, even if this expected value was not near a predicted failure condition. The 
incredible amount of additional analysis and testing performed by the DRL team (led by 
Adam Steltzner) was the most remarkable example of this. In many cases, data was 
validated by an independent means (e.g., chute deploy attitude was computed 
independently from quaternion and accelerometer data). The tendency to blame 
anomalies on the atmosphere (since there was no direct atmospheric measurement) was 
fought until all other system performance possibilities were examined. Each day’s results 
were well calibrated and re-planned through two complete team meetings run by Wayne 
Lee and Rob Manning. The focus of these meetings and prioritization of key findings 
improved each day. The red/yellow/green parameter performance table compiled and 
discussed at these meetings was a very useful way to summarize present status and 
identify remaining priorities. 


RED TEAM’S LIST OF MAJOR EDL ISSUES 

The Red Team identified the following major issues as it assessed the Project’s EDL 
reconstruction process. These issues correlate well with the Reconstruction Team’s 
“yellow” EDL performance items. We, as well as the Reconstruction Team, found no 
showstoppers, or “red” performance issues. (ISA numbers are shown if known.) 

• Reconstructed atmosphere density was lower prior to parachute deploy and higher 
after parachute deploy than predicted by the model used 

Parachute deployment time 2 sigma late (ISA Z83004) 

• DRL over performed ( ISA Z83006) 

• Fair amount of vertical velocity at bridle cut - (wind gust near RAD fire?) (ISA 
Z83010) 

• Unexplained results of the parachute algorithm in last 2 sec 

• Higher than expected attitude of lander (angle of attack) between 175 thru 25 1 sec 
after entry (just prior to chute deploy) (ISA Z83007 ) 

• Undesirable performance of peak width check in DIMES algorithm 

• LCP - RF signal observed during bouncing (ISA Z83009) 

• Various pyro current spikes 

• Various chassis voltage spikes 
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SUB-TEAM PERFORMANCE ASSESSMENTS 

The Atmosphere and Entry Dynamics sub-teams functioned well together over the week. 
In particular, a significant amount of discussion was required to bring the atmospheric 
prediction scientists and atmospheric density engineering reconstruction into agreement. 
The ability to complete this task under stringent time constraints is commendable. 

The Flight Mechanics sub-team was well qualified, motivated, and had an appropriate set 
of tools. The data was used properly with cross checks on the validity of the data where 
this was possible. The key issues were identified and the work was properly prioritized. 
Key resources at LaRC were effectively used to address identified issues. Although a 
process for this work was not documented (in the normal mission operations manner) the 
process worked smoothly with the correct interfaces and teaming arrangements 
developed to effectively complete the reconstruction and development activities. The 
team members were very cooperative and respectful within the team and outside the team 
and appropriately incorporated the EDL Red Team in their activities. 

The members of the Parachute sub-team learned a lot about parachutes and, in one Red 
Team member’s opinion, became legitimate peers of the "parachute experts" that dogged 
their footsteps for 2+ years and the parachute designers from industry that built the 
parachutes. Therefore, they had no problem identifying what the primary parachute 
issues were and how to work them. The Red Team member assigned to them suggests 
that he was more of a nuisance than an asset to them; they did an excellent job on their 
own. This team's work with the Atmospheric Scientists and the trajectory people was a 
notable highlight because all three groups gained a better systems perspective of MER-A 
EDL as a result. 

The Navigation Team and Navigation Advisory Group (NAG) have done an excellent job 
of conducting the approach navigation effort on the MER mission and the EDL effort. 

The team has recognized, utilized, and made significant advances on the processes, 
procedures and tools established by earlier missions. The Navigation team looks to be 
fully engaged with the Project and the relevant spacecraft subsystems, as well as the EDL 
team, ensuring a confident level of communications and interface between the Navigation 
team and the Flight team. 

The telecom support to the EDL Reconstruction Team was viewed as an adjunct activity 
and not as part of the main line EDL reconstruction process. The Reconstruction Team 
got away with this because the MER-A EDL was successful and the semaphore analyses 
were not particularly valuable in the overall context of having a large amount of relay 
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data. But keeping telecom on the perimeter of the EDL team would have been a severe 
encumbrance had there been a mishap during EDL. 


CONCURRENCE WITH IMPACTS TO MER-B EDL 

The Red Team concurs with the two major recommendations made by the Reconstruction 
Team. First, that no “shallowing” maneuver is necessary for MER-B, and, second, that 
timeline margin, if needed, can be safety obtained by increasing the dynamic pressure at 
which the parachute is deployed from 725 Pa (used on MER-A) to 750 Pa. 


SUGGESTIONS FOR FUTURE RECONSTRUCTION ACTIVITIES 

While we found the MER reconstruction activities to have been very well executed, we 
could not help but offer several suggestions for process improvement the next time this 
activity is executed. 

Relative to atmospheric reconstruction, it is suggested that the Project was remiss in not 
flying pressure and temperature sensors for use during terminal descent and on the 
ground. Such sensors, with a likely mass of hundreds of grams, would have allowed 
separation of the density and drag coefficient errors sources, something for which the 
Reconstruction Team could only speculate. It is suggested that these sensors be added to 
the EDL reconstruction requirements of future landed systems. 

A suggested process improvement is the development of a software tool to visually 
display the EDL timeline. Multiple variables could be plotted and displayed 
simultaneously as a function of trajectory variables. For example, it would have been 
useful to see the accelerometer data plotted as a function of time along with entry speed, 
Mach number and altitude. EDL events could also be noted on the timeline. A simple 
switch would then permit plotting of the variables as a function of altitude, for example. 
In this way, a more comprehensive and integrated view of the EDL sequence could be 
communicated. Another process improvement would be an automated frequency 
decomposition of the data. Several times we witnessed a frequency analysis by "eye- 
balling" the data. Although eyeballing is sufficient for the dominant modes, it does not 
reveal power in other parts of the spectrum. Stacking up multiple data traces, denoting 
EDL events, and showing frequency content would be a powerful analysis approach. 
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We all felt the reconstruction process was hindered by the lack of a reference timeline 
(table or schematic) that could be referenced when needed. It was asserted that everyone 
had these numbers in their heads, but in the decision meetings it was clear that they did 
not. 

One of our earlier PORT 10 concerns was the lack of clear decision criteria for change 
that was still evident in the MER-A reconstruction process. 

Because the analysis of the semaphore data may become a significant tool given an EDL 
anomaly in the future, every effort should be made to integrate telecom fully within the 
Reconstruction Team. 


CONCLUSIONS 

The Red Team finds that the MER-A reconstruction was well executed, that the proper 
issues were analyzed, and proper recommendations were made relative to modifications 
for the MER-B EDL. 

The Red Team found its assessment of MER- A EDL reconstruction process to be a 
stimulating assignment and feels quite honored to have associated with such a fine 
engineering group as the MER EDL Reconstruction Team. 
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